A New Adaptive Sampling Method for Scalable Learning

Scaling up data mining algorithms to handle huge data sets is an important issue in machine learning and knowledge discovery. Random sampling is often used to achieve better scalability in learning from massive amount of data. Adaptive sampling offers advantages over traditional batch sampling methods in that adaptive sampling often uses much lower number of samples and thus better efficiency while assuring guaranteed level of estimation accuracy and confidence. In this paper, we present a new adaptive sampling method for estimating the mean of a Bernoulli variable, along with preliminary theoretical studies of the method. We present empirical simulation results indicating that our method often use significantly lower sample size (i.e., the number of sampled instances) while maintaining competitive accuracy and confidence when compared with batch sampling method. We also briefly outline how to make use of this new sampling method to build a scalable ensemble learning algorithm by Boosting.